Language tasks involving character-level manipulations (e.g., spelling correction, many word games) are challenging for models based in subword tokenization. To address this, we adapt the interchange intervention training method of Geiger et al. (2021) to operate on type-level variables over characters. This allows us to encode robust, position-independent character-level information in the internal representations of subword-based models. We additionally introduce a suite of character-level tasks that systematically vary in their dependence on meaning and sequence-level context. While simple character-level tokenization approaches still perform best on purely form-based tasks like string reversal, our method is superior for more complex tasks that blend form, meaning, and context, such as spelling correction in context and word search games. Our approach also leads to subword-based models with human-intepretable internal representations of characters.

Attention-based neural networks, such as Transformers, have become ubiquitous in numerous applications, including computer vision, natural language processing, and time-series analysis. In all kinds of attention networks, the attention maps are crucial as they encode semantic dependencies between input tokens. However, most existing attention networks perform modeling or reasoning based on representations, wherein the attention maps of different layers are learned separately without explicit interactions. In this paper, we propose a novel and generic evolving attention mechanism, which directly models the evolution of inter-token relationships through a chain of residual convolutional modules. The major motivations are twofold. On the one hand, the attention maps in different layers share transferable knowledge, thus adding a residual connection can facilitate the information flow of inter-token relationships across layers. On the other hand, there is naturally an evolutionary trend among attention maps at different abstraction levels, so it is beneficial to exploit a dedicated convolution-based module to capture this process. Equipped with the proposed mechanism, the convolution-enhanced evolving attention networks achieve superior performance in various applications, including time-series representation, natural language understanding, machine translation, and image classification. Especially on time-series representation tasks, Evolving Attention-enhanced Dilated Convolutional (EA-DC-) Transformer outperforms state-of-the-art models significantly, achieving an average of 17% improvement compared to the best SOTA. To the best of our knowledge, this is the first work that explicitly models the layer-wise evolution of attention maps. Our implementation is available at https://github.com/pkuyym/EvolvingAttention

Recently, the success of pre-training in text domain has been fully extended to vision, audio, and cross-modal scenarios. The proposed pre-training models of different modalities are showing a rising trend of homogeneity in their model structures, which brings the opportunity to implement different pre-training models within a uniform framework. In this paper, we present TencentPretrain, a toolkit supporting pre-training models of different modalities. The core feature of TencentPretrain is the modular design. The toolkit uniformly divides pre-training models into 5 components: embedding, encoder, target embedding, decoder, and target. As almost all of common modules are provided in each component, users can choose the desired modules from different components to build a complete pre-training model. The modular design enables users to efficiently reproduce existing pre-training models or build brand-new one. We test the toolkit on text, vision, and audio benchmarks and show that it can match the performance of the original implementations.

Existing approaches for vision-and-language navigation (VLN) are mainly based on cross-modal reasoning over discrete views. However, this scheme may hamper an agent's spatial and numerical reasoning because of incomplete objects within a single view and duplicate observations across views. A potential solution is mapping discrete views into a unified birds's-eye view, which can aggregate partial and duplicate observations. Existing metric maps could achieve this goal, but they suffer from less expressive semantics (e.g. usually predefined labels) and limited map size, which weakens an agent's language grounding and long-term planning ability. Inspired by the robotics community, we introduce hybrid topo-metric maps into VLN, where a topological map is used for long-term planning and a metric map for short-term reasoning. Beyond mapping with more expressive deep features, we further design a pre-training framework via the hybrid map to learn language-informed map representations, which enhances cross-modal grounding and facilitates the final language-guided navigation goal. Extensive experiments demonstrate the effectiveness of the map-based route for VLN, and the proposed method sets the new state-of-the-art on three VLN benchmarks.

The 1$^{\text{st}}$ Workshop on Maritime Computer Vision (MaCVi) 2023 focused on maritime computer vision for Unmanned Aerial Vehicles (UAV) and Unmanned Surface Vehicle (USV), and organized several subchallenges in this domain: (i) UAV-based Maritime Object Detection, (ii) UAV-based Maritime Object Tracking, (iii) USV-based Maritime Obstacle Segmentation and (iv) USV-based Maritime Obstacle Detection. The subchallenges were based on the SeaDronesSee and MODS benchmarks. This report summarizes the main findings of the individual subchallenges and introduces a new benchmark, called SeaDronesSee Object Detection v2, which extends the previous benchmark by including more classes and footage. We provide statistical and qualitative analyses, and assess trends in the best-performing methodologies of over 130 submissions. The methods are summarized in the appendix. The datasets, evaluation code and the leaderboard are publicly available at https://seadronessee.cs.uni-tuebingen.de/macvi.

Designing and analyzing model-based RL (MBRL) algorithms with guaranteed monotonic improvement has been challenging, mainly due to the interdependence between policy optimization and model learning. Existing discrepancy bounds generally ignore the impacts of model shifts, and their corresponding algorithms are prone to degrade performance by drastic model updating. In this work, we first propose a novel and general theoretical scheme for a non-decreasing performance guarantee of MBRL. Our follow-up derived bounds reveal the relationship between model shifts and performance improvement. These discoveries encourage us to formulate a constrained lower-bound optimization problem to permit the monotonicity of MBRL. A further example demonstrates that learning models from a dynamically-varying number of explorations benefit the eventual returns. Motivated by these analyses, we design a simple but effective algorithm CMLO (Constrained Model-shift Lower-bound Optimization), by introducing an event-triggered mechanism that flexibly determines when to update the model. Experiments show that CMLO surpasses other state-of-the-art methods and produces a boost when various policy optimization methods are employed.

强有力的对手例子是评估和增强深神经网络鲁棒性的关键。流行的对抗性攻击算法使用梯度上升最大化非cave损失函数。但是，每种攻击的性能通常对由于信息不足（仅一个输入示例，几乎没有白色盒子源模型和未知的防御策略）而敏感。因此，精心设计的对抗性示例容易过度拟合源模型，从而将其转移性限制在身份不明的架构上。在本文中，我们提出了多种渐近正态分布攻击（Multianda），这是一种新颖的方法，可以明确表征来自学习分布的对抗性扰动。具体而言，我们通过利用随机梯度上升（SGA）的渐近正态性能（SGA）的优势来近似于扰动，然后将整体策略应用于此过程，以估算高斯混合模型，以更好地探索潜在的优化空间。从学习分布中绘制扰动使我们能够为每个输入生成任何数量的对抗示例。近似后验实质上描述了SGA迭代的固定分布，该分布捕获了局部最佳距离周围的几何信息。因此，从分布中得出的样品可靠地保持转移性。我们提出的方法通过对七个正常训练和七个防御模型进行广泛的实验，超过了对具有或没有防御的深度学习模型的九个最先进的黑盒攻击。

我们提出了第一个统一的框架Unicolor，以支持多种方式的着色，包括无条件和条件性的框架，例如中风，示例，文本，甚至是它们的混合。我们没有为每种类型的条件学习单独的模型，而是引入了一个两阶段的着色框架，以将各种条件纳入单个模型。在第一阶段，多模式条件将转换为提示点的共同表示。特别是，我们提出了一种基于剪辑的新方法，将文本转换为提示点。在第二阶段，我们提出了一个基于变压器的网络，该网络由Chroma-vqgan和Hybrid-Transformer组成，以生成以提示点为条件的多样化和高质量的着色结果。定性和定量比较都表明，我们的方法在每种控制方式中都优于最先进的方法，并进一步实现了以前不可行的多模式着色。此外，我们设计了一个交互式界面，显示了我们统一框架在实际用法中的有效性，包括自动着色，混合控制着色，局部再现和迭代色彩编辑。我们的代码和型号可在https://luckyhzt.github.io/unicolor上找到。

步态情绪识别在智能系统中起着至关重要的作用。大多数现有方法通过随着时间的推移专注于当地行动来识别情绪。但是，他们忽略了时间域中不同情绪的有效距离是不同的，而且步行过程中的当地行动非常相似。因此，情绪应由全球状态而不是间接的本地行动代表。为了解决这些问题，这项工作通过构建动态的时间接受场并设计多尺度信息聚集以识别情绪，从而在这项工作中介绍了新型的多量表自适应图卷积网络（MSA-GCN）。在我们的模型中，自适应选择性时空图卷积旨在动态选择卷积内核，以获得不同情绪的软时空特征。此外，跨尺度映射融合机制（CSFM）旨在构建自适应邻接矩阵，以增强信息相互作用并降低冗余。与以前的最先进方法相比，所提出的方法在两个公共数据集上实现了最佳性能，将地图提高了2 \％。我们还进行了广泛的消融研究，以显示不同组件在我们的方法中的有效性。

行业分配根据预定义的行业分类系统（ICS）将公司分配给行业，这对于大量关键业务实践至关重要，从公司的运营和战略决策到政府机构的经济分析。三种专家知识对于有效行业分配至关重要：基于定义的知识（即每个行业的专家定义），基于结构的知识（即ICS中指定的行业之间的结构关系）和基于任务的知识（即，域专家执行的事先公司行业任务）。现有的行业分配方法仅利用基于任务的知识来学习将未分配的公司分类为行业的模型，并忽略基于定义和基于结构的知识。此外，这些方法仅考虑已分配了公司的哪个行业，但忽略了基于分配的知识的时间特异性，即在任务发生时。为了解决现有方法的局限性，我们提出了一种新颖的基于深度学习的方法，该方法不仅无缝整合了三种类型的行业分配知识，而且还考虑了基于分配的知识的特定时间。从方法上讲，我们的方法具有两种创新：动态行业表示和分层分配。前者通过通过我们提出的时间和空间聚集机制整合了三种类型的知识，将行业代表为一系列特定时间的向量。后者将行业和公司的表现作为投入，计算将公司分配给不同行业的可能性，并将公司分配给具有最高概率的行业。